EDA
We can first visualize the data after dimension reduction with zinbWave. We then use t-SNE and color the cells with the allen labels. Since we use various values for the k parameter, we plot two representations with the most extreme values of k.

Reduction in the number of clusters

Improvement in ARI


Comparison with Allen subclass
After ARI merging
## ARI
## sc3 0.48
## Monocle 0.52
## seurat 0.52
After a last consensus step

Stopping during the merging

We can see that the ARI may not be the best metric for that comparison. # Seurat ARI Param

LS0tCmF1dGhvcjogIkhlY3RvciBSb3V4IGRlIELDqXppZXV4IgpkYXRlOiAnYHIgZm9ybWF0KFN5cy50aW1lKCksICIlZCAlQiAsICVZIilgJwpvdXRwdXQ6CiAgaHRtbF9kb2N1bWVudDoKICAgIHRvYzogdHJ1ZQogICAgdG9jX2RlcHRoOiAyCiAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUKICAgIGNvZGVfZG93bmxvYWQ6IFRSVUUKICAgIApwYXJhbXM6CiAgZGF0YXNldDogU01BUlRlcl9udWNsZWlfTU9wIAogIHRpdGxlOiAiQW5hbHlzaXMgb2YgdGhlIFNNQVJUZXJfbnVjbGVpX01PcCBkYXRhc2V0IgotLS0KLS0tCnRpdGxlOiBgciBwYXJhbXMkdGl0bGVgCi0tLQoKYGBge3IgbG9hZCBwYWNrYWdlcywgaW5jbHVkZT1GfQpsaWJyYXJ5KGtuaXRyKQpvcHRzX2NodW5rJHNldCgKICBmaWcucG9zID0gIiFoIiwgb3V0LmV4dHJhID0gIiIsIHdhcm5pbmcgPSBGLCBtZXNzYWdlID0gRiwKICBmaWcud2lkdGggPSA1LCBmaWcuYWxpZ24gPSAiY2VudGVyIiwgZWNobyA9IEYKKQpsaWJzIDwtIGMoImhlcmUiLCAiZHBseXIiLCAiZ2dwbG90MiIsICJ0aWR5ciIsICJzdHJpbmdyIiwgInJlYWRyIiwgImNvd3Bsb3QiLAogICAgICAgICAgImNsdXN0ZXJFeHBlcmltZW50IiwgIm1jbHVzdCIsICJSQ29sb3JCcmV3ZXIiLCAicHJvZ3Jlc3MiLCAibWVyZ2VyIiwKICAgICAgICAgICJwbmciKQpzdXBwcmVzc01lc3NhZ2VzKAogIHN1cHByZXNzV2FybmluZ3Moc2FwcGx5KGxpYnMsIHJlcXVpcmUsIGNoYXJhY3Rlci5vbmx5ID0gVFJVRSkpCikKcm0obGlicykKdHlwZSA8LSBmdW5jdGlvbihkYXRhc2V0KSB7CiAgaWYgKHN0cl9kZXRlY3QoZGF0YXNldCwgIlNNQVJUIikpIHJldHVybigiU21hcnQtU2VxIikKICBpZiAoc3RyX2RldGVjdChkYXRhc2V0LCAiMTB4IikpIHJldHVybigiMTBYIikKICBzdG9wKCJUeXBlIHVua25vd24iKQp9CnR5cGUgPC0gdHlwZShwYXJhbXMkZGF0YXNldCkKbWVyZ2Vyc19ub19hbGxlbiA8LSByZWFkUkRTKGhlcmUoImRhdGEiLCB0eXBlLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwYXN0ZTAocGFyYW1zJGRhdGFzZXQsICJfbm9fYWxsZW5fbWVyZ2Vycy5yZHMiKSkpCmBgYAoKIyBFREEKCldlIGNhbiBmaXJzdCB2aXN1YWxpemUgdGhlIGRhdGEgYWZ0ZXIgZGltZW5zaW9uIHJlZHVjdGlvbiB3aXRoIHppbmJXYXZlLiBXZSB0aGVuIHVzZSB0LVNORSBhbmQgY29sb3IgdGhlIGNlbGxzIHdpdGggdGhlIGFsbGVuIGxhYmVscy4gU2luY2Ugd2UgdXNlIHZhcmlvdXMgdmFsdWVzIGZvciB0aGUgayBwYXJhbWV0ZXIsIHdlIHBsb3QgdHdvIHJlcHJlc2VudGF0aW9ucyB3aXRoIHRoZSBtb3N0IGV4dHJlbWUgdmFsdWVzIG9mIGsuCgpgYGB7ciB0LVNORSBwbG90cywgZmlnLndpZHRoPTE4LCBmaWcuaGVpZ2h0PTl9CnBsb3RzIDwtIGxpc3QuZmlsZXMoaGVyZSgiRmlndXJlcyIsICJFREEiKSkKcGxvdHMgPC0gcGxvdHNbc3RyX2RldGVjdChwbG90cywgcGFyYW1zJGRhdGFzZXQpXQpwbG90Lm5ldygpCnBsb3Qud2luZG93KDA6MSwgMDoxKQpyYXN0ZXJJbWFnZShyZWFkUE5HKGhlcmUoIkZpZ3VyZXMiLCAiRURBIiwgcGxvdHNbMV0pKSwgMCwgMCwgLjUsIDEpCnJhc3RlckltYWdlKHJlYWRQTkcoaGVyZSgiRmlndXJlcyIsICJFREEiLCBwbG90c1tsZW5ndGgocGxvdHMpXSkpLCAwLjUsIDAsIDEsIDEpCmBgYAoKCiMgR2VuZXJhbCBjb21tZW50cwoKV2UgaGF2ZSB0aGUgZm9sbG93aW5nIHdvcmtmbG93CgpgYGB7ciB3b3JrZmxvdywgZmlnLmhlaWdodD02LCBmaWcud2lkdGg9MTB9CnBsb3QubmV3KCkKcGxvdC53aW5kb3coMDoxLCAwOjEpCnJhc3RlckltYWdlKHJlYWRQTkcoaGVyZSgiRmlndXJlcyIsICJTTUFSVC1TZXFfd29ya2Zsb3cucG5nIikpLCAwLCAwLCAxLCAxKQpgYGAKCkluIHRoZSBTZXVyYXQgY2x1c3RlcmluZywgdGhlcmUgYXJlIHR3byBwYXJhbWV0ZXJzIHRvIGNob29zZSBmcm9tOiB0aGUgcmVzb2x1dGlvbiBhbmQgdGhlIGsucGFyYW0gKHVzZSBpbiBhIGtubiBzdGVwKS4gRnJvbSB0aGUgc2V1cmF0IGhlbHAgZmlsZSwgZm9yIHRoZSByZXNvbHV0aW9uIHBhcmFtZXRlcjogdXNlIGEgdmFsdWUgYWJvdmUgKGJlbG93KSAxLjAgaWYgeW91IHdhbnQgdG8gb2J0YWluIGEgbGFyZ2VyIChzbWFsbGVyKSBudW1iZXIgb2YgY29tbXVuaXRpZXMuCgpXZSBwaWNrZWQgYSB2YWx1ZSBvZiAxLjYgYXMgd2Ugd2FudCBtYW55IGNsdXN0ZXJzIHRvIHN0YXJ0IHdpdGguIFdlIGFsc28gcGljayBrID0gNTAuIFRob3NlIHZhbHVlcyBzZWVtIGludGVybWVkaWF0ZSBpbiB0ZXJtIG9mIEFSSSB3aXRoIG90aGVyIHBhaXJzIG9mIHBhcmFtdGVycyAoc2VlIHNlY3Rpb24gU2V1cmF0IEFSSSBwYXJhbSkKCldlIGFsc28gY29uc2lkZXIgdHdvIHdheXMgb2YgY29tcHV0aW5nIHRoZSBBUkkgYmV0d2VlbiBSU0VDIGFuZCBvdGhlciBtZXRob2RzLiBFaXRoZXIsIHdlIGZvcmNlIFJTRUMgdG8gYXNzaWduIGFsbCBjZWxscyB0byBhIGdpdmVuIGNsdXN0ZXIsIG9yIHdlIG9ubHkgY29tcHV0ZSB0aGUgQVJJIGJldHdlZW4gUlNFQyBhbmQgdGhlIG90aGVyIG1ldGhvZHMgb24gdGhvc2UgY2VsbHMgdGhhdCBSU0VDIGRvIGNsdXN0ZXIuIE5vdGUgdGhhdCBpbiB0aGUgc2Vjb25kIGNhc2UsIG90aGVyIHBhaXJzIG9mIG1ldGhvZHMgYXJlIGNvbXBhcmVkIHVzaW5nIGFsbCBjZWxscy4gV2UgZGVub3RlIGFzIFJzZWNUIHRoZSBjbHVzdGVyIGFzc2lnbmVtZW50IHdoZXJlIGFsbCBjZWxscyBhcmUgYXNzaWduZWQgKGkuZSBSc2VjIFRvdGFsKS4gCgpUaGUgQVJJIG1lcmdpbmcgbWV0aG9kIHdvcmtzIGFzIGZvbGxvdy4gV2UgaXRlcmF0ZSBvdmVyIGFsbCBwYWlycyBvZiBjbHVzdGVycyBmb3IgZXZlcnkgY2x1c3RlcmluZyBtZXRob2QuIEZvciBlYWNoIHBhaXIsIHdlIHRyeSB0byBtZXJnZSB0aGUgdHdvIGNsdXN0ZXJzIGFuZCBzZWUgaG93IGl0IGltcHJvdmVzIHRoZSBBUkkgd2l0aCB0aGUgb3RoZXIgbWV0aG9kcy4gV2UgdGhlbiBtZXJnZSB0aGUgcGFpciB0aGF0IGltcHJvdmVzIHRoZSBBUkkgdGhlIG1vc3QuIFdlIHN0b3Agd2hlbiB0aGUgQVJJIGNhbm5vdCBiZSBpbXByb3ZlZCBhbnltb3JlLgoKSW4gdGhlIGdlbmVyYWwgY2FzZSwgd2UgcGVyZm9ybSB0aGUgQVJJIG1lcmdpbmcgd2l0aG91dCB0aGUgYWxsZW4gbGFiZWxzIGFuZCB3ZSB0aGVuIHVzZSBpdCBhcyBhIGNvbXBhcmlzb24uIEEgcXVpY2sgb3ZlcnZpZXcgb2YgaG93IHRoZSBhbGdvcml0aG0gcGVyZm9ybXMgd2l0aCB0aGUgYWxsZW4gbGFiZWxzIGlzIHNlZW4gYXQgdGhlIGVuZC4KCiMgUmVkdWN0aW9uIGluIHRoZSBudW1iZXIgb2YgY2x1c3RlcnMKCmBgYHtyIEltcCBubyBhbGxlbn0KcGxvdFByZVBvc3QobWVyZ2Vyc19ub19hbGxlbikKYGBgCgojIEltcHJvdmVtZW50IGluIEFSSQoKYGBge3IgQVJJIGltcCBubyBhbGxlbiBjZWxsfQpwbG90QVJJUmVkdWNlKG1lcmdlcnNfbm9fYWxsZW4pCmBgYAoKCmBgYHtyIEFSSSB0cmVuZCwgZmlnLndpZHRoPTl9CkFSSXRyZW5kKG1lcmdlciA9IG1lcmdlcnNfbm9fYWxsZW4pCmBgYAoKIyBDb21wYXJpc29uIHdpdGggQWxsZW4gc3ViY2xhc3MKCiMjIEFmdGVyIEFSSSBtZXJnaW5nCgpgYGB7ciBjb21wIGNlbGx9CmFsbGVuX2NsdXN0ZXJzIDwtIHJlYWQuY3N2KGhlcmUoImRhdGEiLCB0eXBlLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBhc3RlMChwYXJhbXMkZGF0YXNldCwgIl9jbHVzdGVyLm1lbWJlcnNoaXAuY3N2IikpLAogICAgICAgICAgICAgICAgICAgICAgICAgICBjb2wubmFtZXMgPSBjKCJjZWxscyIsICJjbHVzdGVyX2lkIikpCmNsdXN0ZXJzIDwtIHJlYWQuY3N2KGhlcmUoImRhdGEiLCB0eXBlLAogICAgICAgICAgICAgICAgICAgICAgICAgIHBhc3RlMChwYXJhbXMkZGF0YXNldCwgIl9jbHVzdGVyLmFubm90YXRpb24uY3N2IikpLAogICAgICAgICAgICAgICAgICAgICBoZWFkZXIgPSBUKQphbGxlbl9jbHVzdGVycyA8LSBmdWxsX2pvaW4oYWxsZW5fY2x1c3RlcnMsIGNsdXN0ZXJzKQpybShjbHVzdGVycykKCmFwcGx5KG1lcmdlcnNfbm9fYWxsZW4kY3VycmVudE1hdCwgMiwgZnVuY3Rpb24oeCkgewogICAgaW5kcyA8LSB4ICE9IC0xCiAgICB4YSA8LSB4W2luZHNdCiAgICB5IDwtIGFsbGVuX2NsdXN0ZXJzJHN1YmNsYXNzX2xhYmVsW2luZHNdCiAgICBtY2x1c3Q6OmFkanVzdGVkUmFuZEluZGV4KHhhLCB5KQp9KSAlPiUgZGF0YS5mcmFtZShBUkkgPSByb3VuZCguLCAyKSkgJT4lCiAgc2VsZWN0KEFSSSkKYGBgCgojIyBBZnRlciBhIGxhc3QgY29uc2Vuc3VzIHN0ZXAKCmBgYHtyIGZpbmFsIGNvbnNlbnN1c30KY2x1c3RlcnMgPC0gbWVyZ2Vyc19ub19hbGxlbiRjdXJyZW50TWF0CnIgPC0gd2hpY2goY29sbmFtZXMoY2x1c3RlcnMpID09ICJSc2VjVCIpCmlmIChhbGwuZXF1YWwoaW50ZWdlcigwKSAscikgIT0gVFJVRSkgewogIGNsdXN0ZXJzWywiUnNlYyJdIDwtIGFzc2lnblJzZWMobWVyZ2Vyc19ub19hbGxlbikgCn0KY2x1c3RlcnMgPC0gYXMubWF0cml4KGNsdXN0ZXJzKSAKCmNlbGxzQ29uc2Vuc3VzIDwtIENvbnNlbnN1cyhjbHVzTWF0ID0gY2x1c3RlcnMsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICBsYXJnZSA9ICh0eXBlICE9ICJTbWFydC1TZXEiKSkKY29uc2Vuc3VzIDwtIGNiaW5kKGNlbGxzQ29uc2Vuc3VzLAogICAgICAgICAgICAgICAgICAgYWxsZW5fY2x1c3RlcnMkc3ViY2xhc3NfbGFiZWwsCiAgICAgICAgICAgICAgICAgICBjbHVzdGVycykKCmNvbG5hbWVzKGNvbnNlbnN1cylbYygxLCAyKV0gPC0gYygiQ29uc2Vuc3VzIiwgIkFsbGVuXG5zdWJjbGFzcyIpCnBsb3RDbHVzdGVycyhvYmplY3QgPSBjb25zZW5zdXMpCnRpdGxlKCJDb25zZW5zdXMgTWVyZ2luZyB3aXRoIHByb3BvcnRpb25cbmFuZCBjb21wYXJpc29uIHdpdGggdGhlIGFsbGVuIHN1YmNsYXNzLiIpCgpybShjbHVzdGVycywgY2VsbHNDb25zZW5zdXMsIGNvbnNlbnN1cykKYGBgCgojIyBTdG9wcGluZyBkdXJpbmcgdGhlIG1lcmdpbmcKCmBgYHtyIGNvbnNlbnN1cyB3aXRoIGludGVybWVkaWF0ZX0KbWlkTWF0IDwtIGludGVybWVkaWF0ZU1hdChtZXJnZXIgPSBtZXJnZXJzX25vX2FsbGVuLAogICAgICAgICAgICAgICAgICAgICAgICAgIHAgPSAuOSkKciA8LSB3aGljaChjb2xuYW1lcyhtaWRNYXQpID09ICJSc2VjVCIpCmlmIChhbGwuZXF1YWwoaW50ZWdlcigwKSAscikgIT0gVFJVRSkgewogIG1pZE1hdFssIlJzZWMiXSA8LSBhc3NpZ25Sc2VjKG1lcmdlcnNfbm9fYWxsZW4sIHAgPSAuOSkKfQptaWRNYXQgPC0gYXMubWF0cml4KG1pZE1hdCkKCmNlbGxzQ29uc2Vuc3VzIDwtIENvbnNlbnN1cyhjbHVzTWF0ID0gbWlkTWF0LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgbGFyZ2UgPSAodHlwZSAhPSAiU21hcnQtU2VxIikpCmNvbnNlbnN1cyA8LSBjYmluZChjZWxsc0NvbnNlbnN1cywKICAgICAgICAgICAgICAgICAgIGFsbGVuX2NsdXN0ZXJzJHN1YmNsYXNzX2xhYmVsLAogICAgICAgICAgICAgICAgICAgbWlkTWF0KSAlPiUgYXMuZGF0YS5mcmFtZSgpCgpjb2xuYW1lcyhjb25zZW5zdXMpW2MoMSwgMildIDwtIGMoIkNvbnNlbnN1cyIsICJBbGxlblxuc3ViY2xhc3MiKQpwbG90Q2x1c3RlcnMob2JqZWN0ID0gY29uc2Vuc3VzICU+JSBhcy5tYXRyaXgoKSkKdGl0bGUoIkNvbnNlbnN1cyBNZXJnaW5nIHdoZW4gc3RvcHBpbmcgZWFybHkiKQoKcm0oY2VsbHNDb25zZW5zdXMsIGNvbnNlbnN1cywgbWlkTWF0KQpgYGAKCmBgYHtyIHBsb3QsIGV2YWwgPSBGQUxTRX0KSW1wQVJJIDwtIEFSSXRyZW5kQWxsZW4obWVyZ2VyID0gbWVyZ2Vyc19ub19hbGxlbiwKICAgICAgICAgICAgICAgICAgICAgICAgYWxsZW4xID0gYWxsZW5fY2x1c3RlcnMkY2x1c3Rlcl9sYWJlbCwKICAgICAgICAgICAgICAgICAgICAgICAgYWxsZW4yID0gYWxsZW5fY2x1c3RlcnMkc3ViY2xhc3NfbGFiZWwsCiAgICAgICAgICAgICAgICAgICAgICAgIHZlcmJvc2UgPSBUUlVFKQoKZ2dwbG90KGRhdGEuZnJhbWUoc3RlcCA9IDA6bnJvdyhtZXJnZXJzX25vX2FsbGVuJG1lcmdlcyksCiAgICAgICAgICAgICAgICAgIHN1YmNsYXNzQWxsZW4gPSBJbXBBUklbLCAyXSwKICAgICAgICAgICAgICAgICAgY2x1c3RlckFsbGVuID0gSW1wQVJJWywgMV0pICU+JQogICAgICAgICBnYXRoZXIoa2V5ID0gInR5cGUiLCB2YWx1ZSA9ICJJbXBBUkkiLCAtc3RlcCksCiAgICAgICBhZXMoeCA9IHN0ZXAsIHkgPSBJbXBBUkkpKSArCiAgZ2VvbV9saW5lKGFlcyhjb2wgPSB0eXBlKSkgKwogIHNjYWxlX3lfY29udGludW91cyhsaW1pdHMgPSBjKDAsIDEpKQpgYGAKCldlIGNhbiBzZWUgdGhhdCB0aGUgQVJJIG1heSBub3QgYmUgdGhlIGJlc3QgbWV0cmljIGZvciB0aGF0IGNvbXBhcmlzb24uCiMgU2V1cmF0IEFSSSBQYXJhbQoKYGBge3Igc2V1cmF0IHBhcmFtfQpwbG90Lm5ldygpCnBsb3Qud2luZG93KDA6MSwgMDoxKQpyYXN0ZXJJbWFnZShyZWFkUE5HKAogIGhlcmUoIkZpZ3VyZXMiLCB0eXBlLCBwYXN0ZTAocGFyYW1zJGRhdGFzZXQsICJfc2V1cmF0X0FSSS5wbmciKSkpLAogIDAsIDAsIDEsIDEpCmBgYAo=